Feature Tracking on Occluding Boundaries for Modeling-by-videotaping

This paper proposes a novel method to deal with the tracking of feature points on occluding boundaries. It can distinguish the feature points on occluding boundaries from those inside the boundaries. It can detect the disappearance of a feature point caused by occlusion during the tracking. It can extract a newly appeared feature point suitable to tracking. For the purpose, we combined an adaptive region-based monitoring process with the ordinary block-based tracking process. When the monitoring process decides that the result of the block-based motion estimation is ambiguous, the motion is adjusted by using the region-based estimation. The decision is made based on the similarity measure of the regions in the blocks in succeeding frames. In order to use the factorization method to reconstruct the 3D shape from the tracking results which includes many blanks caused by the occlusion, we investigated a method which iterates the solution propagation among the 3D and the 2D coordinates of feature points and the camera parameters. It can ll in the unobserved feature coordinates from the observed ones obtained by tracking. We applied the method to several scenes obtained by a hand-held video camera.